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The primary objective of this work was to develop and demonstrate a process for ac- 
curate and efficient uncertainty quantification and certification prediction of low-boom, 
supersonic, transport aircraft. High-fidelity computational fluid dynamics models of multi- 
ple low-boom configurations were investigated including the Lockheed Martin SEEB-ALR 
body of revolution, the NASA 69° Delta Wing, and the Lockheed Martin 1021-01 configu- 
ration. A nonintrusive polynomial chaos surrogate modeling approach was used for reduced 
computational cost of propagating mixed, inherent (aleatory) and model-form (epistemic) 
uncertainty from both the computation fluid dynamics model and the near-field to ground 
level propagation model. A methodology has also been introduced to quantify the plausi- 
bility of a design to pass a certification under uncertainty. Results of this study include 
the analysis of each of the three configurations of interest under inviscid and fully turbu- 
lent flow assumptions. A comparison of the uncertainty outputs and sensitivity analyses 
between the configurations is also given. The results of this study illustrate the flexibility 
and robustness of the developed framework as a tool for uncertainty quantification and 
certification prediction of low-boom, supersonic aircraft. 


Nomenclature 


N s 

Number of Samples 


Random Basis Function 

N t 

Number of Terms in a Total-Order 

t 

Standard Input Random Variable 


Polynomial Chaos Expansion 

a 

Deterministic Coefficient in the Polynomial 

n 

Number of Random Dimensions 


Chaos Expansion 

V 

Polynomial Order 

CR 

Confidence Ratio 

cm* 

Stochastic Response 




I. Introduction 

One of the most substantial obstacles to commercial supersonic flight over land is the creation of a sonic 
boom. Current Federal Aviation Administration and Committee on Aviation Environmental Protection 
restrictions prohibit commercial supersonic flight over many countries, including the United States, due to 
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the ground level boom signature. Recent thrusts in research have focused on reducing the boom signature in 
hopes of reversing the regulations, in turn increasing the viability of supersonic flight. In 2003, after a series 
of tests that measured the ground signature of a F-5 Shaped Sonic Boom Demonstration (SSBD) supersonic 
aircraft, the SSBD program concluded sonic boom mitigation was possible through shape optimization. 1 This 
has launched programs focusing on the design of low boom aircraft. Some examples are Gulfstream’s Quiet 
Spike, Lockheed Martin’s and Boeing’s N+l and N+2 supersonic designs, and Japan Aerospace Exploration 
Agency’s (JAXA) NEXST and S3TD programs. 2-5 In 2008, NASA held a Sonic Boom Prediction Workshop 
to assess the performance of computational fluid dynamics (CFD) for near-field boom prediction and a 2014 
conference recently revisited this topic. 6 

As a means of aiding and accelerating the design process, CFD has become an important part of de- 
veloping supersonic aircraft. Current, state of the art predictions use a three-dimensional CFD solvers to 
determine the near-field pressure signature, which is then numerically propagated to the ground level. These 
numerical results are often used to help inform the design of supersonic vehicles through various techniques, 
such as adjoint-based optimization employing near-field targeting or ground signature loudness 7-9 and less 
computationally demanding derivative-free techniques. 10 In any case, these computational models take one 
set of inputs (which may include the geometry, freestream conditions, angle-of-attack, or modeling parame- 
ters for turbulence and boom propagation) and produce a deterministic set of outputs (e.g., ground signature, 
final optimized geometry, and near-field flow information). Often, the input parameters may not be known 
exactly due to inherent variations in flight conditions and manufacturing tolerances. Additionally, error is 
introduced through assumptions made when creating the relevant mathematical models. Supersonic aircraft 
are sensitive to these changes and this motivates the development of a framework which can account for and 
efficiently represent the uncertainties in the design space for use in reliable and robust design of supersonic 
low-boom configurations that will meet necessary performance standards. 

The primary objective of this study is to develop and demonstrate a framework for accurate and effi- 
cient uncertainty quantification (UQ) and certification prediction methodology for low-boonr configurations. 
Because of the computational demand of low-boom configuration models, traditional sampling techniques, 
such as Monte Carlo, may not be feasible. The goal is to minimize the number of computationally expensive 
deterministic model evaluations needed for an accurate UQ analysis. In this study, a non-intrusive polyno- 
mial chaos surrogate modeling approach is used to represent the CFD and boom propagation models for 
each low-boom configuration investigated. Uncertainty in low-boonr configuration modeling included mixed 
model-form (epistemic) and inherent (aleatory) uncertainty, which are propagated through the surrogate 
models. Three geometries were identified to be of interest and are used as demonstration configurations of 
the developed framework. The configurations selected were an axisymmetric, body of revolution (BOR) from 
Lockheed-Martin (SEEB), the NASA 69° Delta Wing, and the Lockheed Martin (LM) 1021-01 low-boonr 
aircraft configuration. 

The following section of this paper introduces the computational modeling of sonic boom signatures, 
as well as how signatures predicted near the configuration are propagated to the ground. Section III then 
describes the UQ methodology employed, which included the creation of surrogate models for the output 
quantities of interest. A brief description of the types of uncertainty and the propagation of mixed uncertainty 
are given in Section IV. Section V describes the methodology utilized for certification prediction of selected 
performance metrics for low-boom configurations. The UQ and certification prediction framework is then 
summarized in Section VI and demonstrated in Section VII using three low-boom configurations. Important 
conclusions obtained from this study are summarized in the final section. 

II. Modeling and Propagation of Sonic Booms 

In this study, a high-fidelity approach was used to model sonic booms produced by low-boom configura- 
tions. This involved a high resolution CFD analysis near the body (the near- field). The near-field domain 
includes the vehicle and a region extending multiple body lengths away from the aircraft. The goal is to 
resolve the near-field pressure signature, parallel to the flow direction, that is generated by the body in 
supersonic flow. An example of this is shown in Figure 1. This signature is then propagated to ground 
level using a high fidelity propagation code called sBOOM. 9 This simulates how the signal will change while 
passing through the atmosphere. Once a final ground signature is predicted, output quantities of interest, 
such as perceived loudness (PLdB) and C-weighted sound exposure level (CSEL), are evaluated and may 
serve as design and certification metrics. The remainder of this section gives further detail regarding the 
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CFD simulations and the propagation model. 



Figure 1: Near-Field CFD Domain with Pressure Signature 


A. Computational Fluid Dynamics Modeling 

This study employed the Fully Unstructured Navier-Stokes Three-Dimensional (FUN3D) 11 flow solver for 
CFD analysis of low-boonr configurations. FUN3D contains many tools for design optimization and analysis 
and was developed and is maintained at the NASA Langley Research Center. The code employs the finite 
volume formulation and stores flow field variables at the control volume nodes. FUN3D can handle aerody- 
namic simulations across a large range of Mach numbers from the subsonic to hypersonic regimes and has 
been used extensively for the analysis of sonic boom prediction. 12-14 

All cases investigated in this study were assumed to be at steady state. Often, the Euler equations 
are solved in place of the full Navier-Stokes equations for high Reynolds number, aerodynamic flows by 
assuming the flow is inviscid. While this is a known simplification of the actual flow physics, it allows for 
computational savings when investigating large-scale-complex problems or when large numbers of simulations 
are needed, such as when performing UQ. However, low-boom configurations may be designed to exploit 
viscous effects, which smear or dampen shocks. This would be overlooked when using an Euler assumption. 
In the current study, both inviscid and viscous, fully turbulent cases were explored. When solving the Euler 
equations, the inviscid fluxes were calculated at cell edges by employing the van Leer scheme to solve an 
approximate Riemann problem. For viscous cases, the inviscid fluxes are calculated by the Roe scheme 
for the approximate Riemann problem, if possible. In order to aid convergence, the viscous simulations of 
the LM-1021 and 69° Delta wing required the use of the dissipative Low Diffusion Flux Splitting scheme. 
For modeling the turbulence, the one equation Spalart-Allmaras 15 model was employed for computational 
efficiency and robustness. 

B. Computational Grids 

The configurations of interest (SEEB-ALR, 69° Delta Wing, and the LM 1021-01) are shown in Figure 2. 
The SEEB-ALR model is described by Morgenstern et al. 3 and the 69° Delta Wing is discussed by Hunton 
et al. 16 The LM 1021-01 low-boonr configuration is detailed by Morgenstern et al. 17 The discussion of the 
physical models, their origins, and design specific details are left to these studies. 

The inviscid SEEB-ALR and inviscid Delta Wing grids were the same grids made available for the 2014 
AIAA Sonic Boom Prediction Workshop. The SEEB-ALR and Delta Wing models began as STEP files and 
a solid representation was extracted. Then, a triangular surface mesh and tetrahedral volume mesh were 
generated using GridEx. 19 The core grid was then extruded using the Inflate method 13 in a direction aligned 
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Figure 2: Configurations Studied in the Present Work. SEEB-ALR (left), NASA 69° Delta 
Wing (center), LM 1020-01 (right). Models Not to Relative Scale. Reproduced with permis- 
sion from Aftosmis et al. 18 


with the Mach angle so the relevant pressure signatures can be captured without wasting points in regions 
which are not influenced by the aircraft. Full details on the SEEB-ALR and 69° Delta Wing solid models, 
grids, and grid generation techniques for near- field sonic boom CFD are given in Park et al. 13 A cross section 
showing both the core grid and part of extruded region for the Delta Wing model is included in Figure 3. 

One point of interest is that two versions of the SEEB-ALR solid model exist: the as-built and the 
as-designed. The as-designed geometry has a perfectly smooth surface, but the as-built model used in the 
wind tunnel testing possessed many surface imperfections as shown in Figure 4. It will later be shown that 
these flaws are reflected in the near-field signature. One objective of this study will be to quantify the effect 
of the surface imperfections on the loudness quantities of interest in this study. 



Figure 3: Computational Grid for the Vis- 
cous 69° Delta Wing Including Extruded Re- 
gion 



Figure 4: SEEB-ALR Surface Imperfections 
of the as-Built Model 
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Table 1 summarizes the grid sizes for each model. In general, viscous grids needed to be much more 
refined. Coupled with the turbulence model, viscous cases took 5-10 times longer to converge than the Euler 
cases. This is especially significant for the LM-1021. All cases were run in parallel over 192 processors. 

Table 1: CFD Grid Dimensions and Computational Time demands 


Model 

Cells 

Nodes 

Solution Time 

SEEB-ALR Euler 

7.83e+06 

2.89e+06 

20 min 

SEEB-ALR Viscous 

3.64e+07 

6.20e+06 

1.3 hrs 

69° Delta Wing Euler 

2.24e+07 

5.36e+06 

10 min 

69° Delta Wing Viscous 

7.80e+06 

2.90e+06 

2 hrs 

LM 1021-01 Euler 

3.38e+07 

8.37e+06 

30 min 

LM 1021-01 Viscous 

7.24e+07 

2.45e+07 

5 hrs 


To lessen the computational burden of the multiple CFD evaluations required for the UQ analysis, baseline 
cases with unperturbed parameter values were completed for each vehicle. These were then restarted for each 
random sample vector within the stochastic parameter space. For every configuration, this resulted in fewer 
iterations to achieve convergence compared to a completely new solution. This was especially significant for 
the LM-1021. 

C. Near-field Signature Propagation 

After obtaining the near-field pressure signature from the CFD model, it can then be propagated to the 
ground using a program called sBOOM. This model uses an augmented Burgers equation to propagate the 
near-field pressure signature to the ground level. The model takes into account nonlinear effects, thermo- 
viscous absorption, atmospheric stratification, spreading, and many other molecular relaxation phenomena. 
Further details regarding this model are given by Rallabhandi. 20 

III. Uncertainty Quantification Methodology 

Many high-fidelity numerical models, such as those discussed in section II, are computationally demand- 
ing. When performing UQ, traditional sampling methods may be problematic as large numbers of evaluations 
of the deterministic model are typically required for accurate results. In this study, a surrogate modeling 
approach based on polynomial chaos theory is utilized for improved computational efficiency. This section 
details the formulation of the surrogate models for use in accurate and efficient UQ. 

A. Point-Collocation Nonintrusive Polynomial Chaos 

In recent studies, 21-25 the polynomial chaos method has been used as a means of UQ over traditional methods, 
such as Monte Carlo, for its computational efficiency. Polynomial chaos is a surrogate modeling technique 
based on the spectral representation of the uncertainty. An important aspect of spectral representations is 
the decomposition of a response value or random function a* into separable deterministic and stochastic 
components, as shown in Eq. (1). 


p 

a*( x ,£) « 5^a*( x )^»(£) (!) 

i = 0 

Here, ctj is the deterministic component and T, is the random variable basis functions corresponding to 
the i i/l mode, a* is assumed to be a function of the vector x of independent random variables and the 
n-dimensional standard random variable vector Note that this series is, by definition, an infinite series. 
However, in practice, it is truncated and a discrete sum is taken over a number of output modes. 26 To form 
a complete basis or for a total order expansion, N t terms are required, which can be computed from Eq. (2) 
for a polynomial chaos expansion (PCE) of order p and a number of random dimensions or variables, n. 
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N t = P + 1 


( 2 ) 


(n + p)\ 
n\p\ 

Further details on polynomial chaos theory are given by Walters and Huyse, 2 ' Eldred 26 and Ghanem. 28 

The objective with any PCE method is to determine the expansion coefficients, a,;. To do this, polynomial 
chaos methods can be implemented using an intrusive or a non-intrusive approach. While an intrusive method 
may appear straightforward in theory, for complex problems this process may be time consuming, expensive, 
and difficult to implement. 21 In contrast, the non-intrusive approach can be easily implemented to construct 
a surrogate model that represents a complex computational simulation, because no modification to the 
deterministic model is required. The non-intrusive methods require only the response (or sensitivity) 29-31 
values at selected sample points to approximate the stochastic response surface. 

Several methods have been developed for non-intrusive polynomial chaos (NIPC). Of these, the point- 
collocation NIPC method has been used extensively in many aerospace simulations and CFD problems. 22, 23,25,29 
The point-collocation method starts with replacing a stochastic response or random function with its PCE 
using Eq. (1). Then, N t vectors are chosen in random space and the deterministic code is then evaluated at 
these points, which is the left hand side of Eq. (1). Following this, a linear system of N t equations can be 
formulated and solved for the spectral modes of the random variables. This system is shown in Eq. (3). 


/*o(€o) *i(€o) ••• MU>A 




A*( x ,£ qA 

MU) MU) ••• ’MU) 


Oil 

= 

a*(x,U) 

VMM MM ••• MM/ 


\a P J 
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Note that for this linear system, N t is the minimum number of deterministic samples required to obtain 
an analytical solution (i.e. , the coefficient vector). If more samples are available and that are linearly 
independent, the system is considered overdetermined and can be solved using a least squares approach. 
The number of samples over the required minimum is represented by the use of an oversampling ratio 
(OSR), defined as the ratio of number of actual samples to the minimum number required (i.e., N t ). In 
general, the number of collocation points can be determined by multiplying Eq. (2) by an OSR. Hosder et 
al. 32 determined an effective OSR of two for the stochastic model problems studied. It was shown that the 
accuracy of the PCE is dependent on the number of collocation points. 

IV. Types of Uncertainty and Mixed Uncertainty Propagation 

A critical step in any uncertainty analysis is the classification of the uncertain parameters. These pa- 
rameters may be mathematically represented differently based on the nature of their uncertainty. Incorrect 
classification and/or treatment of uncertain parameters can result in widely varying output uncertainty. The 
objective of this section is to describe the two main categories of uncertainty that exist in numerical modeling 
and outline a procedure for propagating uncertainty when both types are present. 

A. Types of Uncertainty 

Two main types of uncertainty exist in numerical modeling: aleatory uncertainty and epistemic uncertainty. 33 
Aleatory uncertainty is the inherent variation of a physical system. Such variation is due to the random 
nature of input data and can be mathematically represented by a probability density function if substantial 
experimental data is available for estimating the distribution type. An example of this for stochastic CFD 
simulations could be the fluctuation in freestream quantities. While still considered a random variable, these 
variables are not controllable and their uncertainty is sometimes referred to as irreducible. 

Epistemic uncertainty in a stochastic problem comes from several potential sources. These include a lack 
of knowledge or incomplete information of the behavior of a particular variable. Also, ignorance or negligence 
with regards to accurate treatment of model parameters is a source of epistemic uncertainty. Contrary to 
aleatory uncertainty, epistemic uncertainty is sometimes referred to as reducible uncertainty. An increase 
in knowledge regarding the physics of a problem, along with accurate modeling, can reduce the amount 
of this type of uncertainty. Epistemic uncertainty is typically modeled using intervals because the use of 
probabilistic distributions (even a uniform distribution) can lead to inaccurate predictions in the amount of 
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uncertainty in a system. Upper and lower bounds of these intervals can be drawn from limited experimental 
data or from expert predictions and judgment. 21,22 

An additional, special case of epistemic uncertainty is numerical error. This uncertainty is common in 
numerical modeling and is defined as a recognizable deficiency in any phase or activity of modeling and 
simulations that is not due to lack of knowledge of the physical system. In CFD, an example of this type 
of uncertainty would be the discretization error in both the temporal and spatial domains that comes from 
the numerical solution of the partial differential equations that govern the system. 22 This uncertainty can 
be well understood and controlled through code verification and grid convergence studies. 

B. Mixed Uncertainty Quantification 

Many stochastic problems, including those to be used in this study, may contain both epistemic and aleatory 
types of uncertainty. The desired approach is to consider the contribution of both types of uncertainty 
simultaneously by propagating the mixed uncertainty through the stochastic model. This can be done using 
a procedure known as second-order probability. An NIPC response surface can be used within second-order 
probability in place of the deterministic code, as shown in Figure 5. Traditionally, second-order probability 
is a type of double loop sampling. In the outer loop, a vector of specific values for the epistemic variables is 
passed into the inner loop where the stochastic response surface resulting from the NIPC analysis is sampled 
for the single epistemic sample vector and every aleatory sample vector. The process is repeated for all of 
the epistemic sample vectors. This means that the total number of samples of the NIPC response surface is 
the number of epistemic samples times the number of aleatory samples. While many samples are typically 
required, In general, sampling the surrogate model is much less computationally expensive than sampling 
the deterministic model. 

Each iteration of the outer loop generates a cumulative distribution function (CDF) based on the aleatory 
uncertainty analysis in the inner loop. After completion of the process, what remains is a series of CDFs, 
which, when plotted, gives intervals of the output variable from the model at different probability levels (i.e. , 
a probability or “P-box” representation of mixed uncertainty output). Second-order probability can also be 
implemented with a Monte Carlo approach that uses the original model in the place of the response surface, 
which is done for the first model problem in this study for comparison with the NIPC results. 



I 


Figure 5: Schematic of Second-Order Probability 

As shown by Eldred and Swiler, 34 the outer (epistemic) loop may also be replaced by optimization- 
based interval size determination approaches, which include both local and global optimization methods 
to determine interval bounds of the output response at selected probability levels. One approach, based 
on the combination of sampling and local optimization, can be used for the outer loop to determine the 
bounds at selected probability levels. The outer loop is first evaluated with a small number of samples to 
determine robust estimates for the initial values of epistemic variables used in local optimization. Then, the 
optimization is performed with these initial starting points for minimizing or maximizing the response at 
selected probability levels. This approach provides a computationally efficient means of obtaining accurate 
results (i.e., interval bounds) via optimization, rather than using extensive sampling of the response surface. 

V. Certification Prediction 

After quantifying the uncertainty in a configuration model, the next step is to determine the reliability 
of the design under the measured uncertainty. As mentioned in Section I, one of the obstacles of commercial 
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supersonic flight is sonic boom production and loudness effect at the ground level. Future regulations on 
commercial supersonic flight may place limitations on sonic boom loudness measures. One objective of this 
study is to outline a procedure for predicting certification potential of supersonic configurations. 

In order to predict the certification plausibility of a particular configuration a process capability analysis 
may be employed. 35 The objective of this analysis is to compare the performance of a process, or in 
this case the performance metrics (e.g., PLdB, CSEL, etc.), against performance specifications or limits 
(i.e., certification values). Several factors must be considered, including a margin, measured between a 
performance metric and a certification value, as well as the uncertainty in a performance metric. For a 
probabilistic representation of the uncertainty in a particular performance metric, any margin measurement 
will also carry a probability. Figure 6 illustrates how a margin can be determined from a P-box obtained 
after the propagation of mixed uncertainty (see Section IV). Notice that the margin is measured between 
highest response value at a selected probability level or requirement (e.g., 95% confidence level) and the 
certification value. A positive margin, shown in Figure 6(a), would exist when the response value (at the 
probability requirement level) is less than the certification value. On the other hand, if the response value 
is greater than the certification value, as in Figure 6(c), the margin would be negative and may indicate the 
design would not pass certification. 





(a) Large Positive Margin (b) Small Positive Margin (c) Negative Margin 

Figure 6: Certification Predication Margin and Uncertainty Measurements 


Because a probability level or requirement is used as part of the margin measurement, it would be possible 
that the margin may be positive, even if the boundary of the P-box hangs over the “Failure Region” of the 
certification value, as shown in Figure 6(b). While the margin is positive, the reliability of the configuration 
performance may be in question. Accounting for this uncertainty is desirable for a reliable design and 
accurate certification prediction. Using the approach outlined by West et al. 36 a ratio between the margin 
and the uncertainty may be used as a reliability measure in the certification prediction. This ratio is termed 
a confidence ratio, CR , and is shown in Eq. (4). 

CR = r M “ r9 ' n (4) 

U ncertainty 

Note that the measurement of the uncertainty is conservatively taken as the distance between the lower 50% 
probability level response value and the upper response value at the probability level of the certification 
predication analysis. West et al. 36 also provides details on the treatment of different types of uncertainty 
representation (i.e., pure aleatory and pure epistemic), as well treatment of cases with uncertainty in the 
certification values. 

This confidence ratio can be viewed as a type of factor of safety. If the margin is larger then the 
uncertainty in the performance metric, the confidence ratio will be greater than one. In the instance when the 
uncertainty is greater than the margin, a CR of less than one will result and may indicate weak reliability in 
the configuration to sufficiently meet a certification requirement. A weak system reliability or failure to meet 
the certification value (see Figure 6(c)) may warrant changes to the configuration modeling (e.g., reduction of 
the epistemic uncertainty or a redesign of the actual configuration) or changes in the certification prediction 
analysis, such as using a lower probability requirement or even reevaluating the certification requirement. 
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VI. Uncertainty Quantification and Certification Prediction Framework 


Previous sections of this study describe methodologies of accurate and efficient UQ and certification 
prediction of low-boom supersonic transport configurations. In order to streamline the analysis, a framework 
using these methodologies was formulated and is shown in a flowchart in Figure 7. Note that the first step 
in this framework is a comparison of the CFD model with experimental data. However, in cases where this 
process is used in tandem with design optimization, experimental data may not exist. 



Figure 7: Uncertainty Quantification and Certification Prediction Process 


Additionally, because of the use of a surrogate model, checking the accuracy of the surrogate models is 
necessary. One approach for measuring the accuracy of a surrogate model is to compare response values 
from the surrogate to a set of sample points distributed throughout the design space, separate from the 
surrogate model training samples. The objective is to measure the error between response values from the 
PCE and the actual deterministic model values at the test sample locations. This approach is an indication 
of the local accuracy of the surrogate model. In this study, sample points from a Latin Hypercube sample 
structure of the design space and an average of the errors is used as an indication of the surrogate accuracy. 
Latin Hypercube sampling has the advantage of providing well distributed coverage of the design space with 
even small sample sizes. Note that each test point used to measure the accuracy of the surrogate model does 
require an additional evaluation of the deterministic model. 

VII. Sources of Uncertainty and Low-Boom Model Analysis Results 

In this section, the UQ and certification prediction framework outlined in the previous section is demon- 
strated on three low-boom configurations: the SEEB-ALR body of revolution, the NASA 69° Delta Wing 
heritage model, and the Lockheed Martin 1021-01 configuration. The section begins with the identification 
of possible sources of uncertainty in the modeling approach and in low-boom configuration operation (i.e. , 
in flight uncertainty). Then, the framework is demonstrated on each of the configurations with detailed 
discussion of output uncertainty in design parameters of interest. Note that for each of the configurations, 
the experimental data used to validate the CFD models was taken from wind tunnel test data discussed in 
detail by Cliff et al. 37 

Additionally, a sensitivity analysis was performed on each configuration to identify the uncertain parame- 
ters that contribute most significantly to the total output variance. This information can be used for multiple 
purposes including targeting resources for improvement of models, refinement of uncertain information, or in 
design. The parameter sensitivities were determined using a global nonlinear approach called Sobol indices 
and is based on the PCE. Further details regarding this approach are given by Sudret, 38 Crestaux et al., 39 
as well as Ghaffari et al. 40 

A. Sources of Uncertainty in Low-Boom Configuration Modeling 

One of the most critical steps in any uncertainty analysis is the identification and classification of the input 
uncertainties of the models. In this study, the deterministic model is decomposed into two parts as outlined 
in section II. First, for the CFD model, two input uncertain parameters were identified as possible sources of 
uncertainty: Mach number and angle of attack. The type and amount of uncertainty are shown in Table 2. 
Note that these parameters were determined to be aleatory parameters given their possible inherent nature 
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as small fluctuations in these parameters are typically unavoidable. For the Mach number, two mean values 
are considered for comparison with the experimental data. The SEEB-ALR and LM 1021-01 configurations 
were tested at Mach 1.6, while the Delta Wing was tested at Mach 1.7. 


Table 2: CFD Aleatory Input Parameters 


Input 

Distribution 

Mean 

Std. Dev. 

Angle of Attack 

Gaussian 

0.0 

0.1 

Mach Number 

Gaussian 

1.6/1. 7 

0.0016 


A second set of uncertain parameters were identified as coming from the propagation model, sBOOM. 20 
This model contained a particularly large amount of uncertainty, both aleatory and epistemic, due to its 
complexity and the large number of tunable input parameters. The aleatory parameters, listed in Table 3 
consist of two groups. The first is uncertainty in the atmosphere that may effect the propagation of the sonic 
boom signature as it travels from the vehicle near-field to the ground. The second group of parameters are 
those that may effect the shape of the signature due to changes in vehicle orientation and orientation rates. 
The measurement location or azimuth is also considered uncertain. In this study, only the on-track position, 
with uncertainty, is investigated. 

Table 3: sBOOM Aleatory Input Parameters 


Input 

Distribution 

Mean 

Std. Dev. 

Temperature Profile (%) 

Gaussian 

1.0 

0.01 

Humidity Profile (%) 

Gaussian 

1.0 

0.01 

Climb Angle (Deg.) 

Gaussian 

0.0 

0.1 

Azimuth (Deg.) 

Gaussian 

0.0 

0.1 

Turn Rate (Deg./s) 

Gaussian 

0.0 

0.05 

Climb Rate (Deg./s) 

Gaussian 

0.0 

0.05 


As with the CFD uncertain parameters, the aleatory parameters are those that have been modeled, 
but random fluctuations may still occur. However, epistemic parameters, listed in Table 4, exist due to 
lack of knowledge of the correct modeling approach. The initial step size and signature propagation points 
parameters are adjustable sBOOM specific input parameters. The signature propagation points variable is 
related to the sampling frequency of the signal and may directly effect the accuracy of the loudness metrics 
from the integrated ground level signature. The number of points necessary for an accurate result may vary 
based on the signal length, signal shape, or the signal source (i.e. , the configuration) . Note also that the 
value of this parameter is both the number of points used within the propagation routine and the number 
of points in the final ground level signature that is analyzed to find the loudness measures. The other two 
parameters (reflection factor and ground level altitude) are considered epistemic uncertain parameters as 
they are not being modeled for a specific flight location. These parameters may vary significantly along a 
flight path. 


Table 4: sBOOM Epistemic Input Parameters 


Input 

Min. 

Max. 

Initial Step Size 

0.007 

0.03 

Reflection Factor 

1.8 

2.0 

Ground Elevation (ft) 

0.0 

5000.0 

Signature Propagation Points 

20000 

60000 
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Iii addition to the above uncertain input parameters, inviscid and fully turbulent CFD solutions were 
performed for all of the configurations analyzed in this study. The purpose of this is to quantify the effect 
of different modeling fidelities for the various low-boom configurations studied. Note that the amount of 
uncertainty in each parameter discussed above was the result of much discussion among the authors and 
other experts in the field. 

B. SEEB-ALR Body of Revolution 

1. Comparison with Experiment 

The first step in the UQ process is to validate the CFD solution. Near-field signatures were taken at 21.2 
inches from the body to coincide with experimental data. Comparison with the experiment is shown in 
Figure 8(a) for both as-built and as-designed geometries, and for both inviscid and fully turbulent flow 
assumptions. A residual scale plot of the signatures is shown in Figure 8(b). In this figure, the averaged 
experimental signature is treated as a reference and is subtracted off of the uncertainty bounds and the 
CFD signatures. Notice that there is good agreement between the experiments and the CFD results as 
the CFD signatures mostly lie within the bounds of the uncertainty in the experimental results. There are 
some differences in the peaks of the signatures, but the experimental results have rounded peaks due to the 
measurement approach. 3 ' This agreement is deemed to be acceptable and is assumed to validate the baseline 
numerical solution. 



— Averaged Experiment 

Experimental Uncertainty Bounds 
— CFD as designed Euler 
— CFD as built Euler 
— CFD as designed Turb-SA 
— CFD as built Turb-SA 


0 


SEEB-ALR h=21 .2 in, phi=0 



(a) Signatures (b) Residual Scale 

Figure 8: SEEB-ALR Near-Field CFD Comparison with Experimental Results 


2. Uncertainty Quantification 

After validating the CFD model, the next step is to construct the surrogate model(s) that represent the 
design quantities of interest. In this study, PLdB and CSEL were considered. In total, there are 12 uncertain 
parameters, as discussed above. Using Eq. (2), 182 evaluations of the deterministic model were necessary to 
construct a second order PCE with twice oversampling. In evaluating the deterministic model, the first step 
is to obtain CFD solutions for the near-field pressure signatures. The dispersion of the deterministic samples 
for the Euler as-built case are shown in Figure 9(a). Figure 9(b) shows the same dispersion of signals, but 
shifted to reference location for better visualization of changes in signature amplitude. 

The CFD signatures were then each propagated through sBOOM with the additional uncertainty that 
was identified for this model. For the Euler as-built case, the resulting ground signature dispersion is shown 
in Figure 10(a) and shifted to a reference location in Figure 10(b). The ground signatures were then analyzed 
to produce the desired loudness quantities of interest. 

Similar dispersions of the near-field and ground level signatures can be generated for the other three 
cases investigated for this model (Euler as-designed, turbulent as-built and turbulent as-designed). While 
not shown here, these dispersions are similar to those shown for the Euler as-built case. 
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(a) Actual (b) Shifted 

Figure 9: SEEB-ALR Euler as-Built Near-Field Pressure Signature Dispersion 




(a) Actual (b) Shifted 

Figure 10: SEEB-ALR Euler as-Built Ground Signature Dispersion 


Note that for most of the CFD models, the signatures do not recover fully. This can stem from the 
inclusion of the mounting system used within the wind tunnel in the computational geometry and/or the 
truncation of the computational domain. For more accurate results and to eliminate any numerical instabil- 
ities within the propagation routine/loudness measure calculations, the ends of the signatures are forced to 
zero and additional zero padding is added to the signatures for all of the configurations in this study. 

At this point, the surrogate models for the output quantities of interest can be created by solving the 
linear system of equations given in Eq. (3). Measuring the accuracy of the surrogate models across the 
design space can be done by sampling a set of test points in the design space and comparing outputs from 
the deterministic model with the outputs from the surrogates at the same sample locations. Twenty new 
samples were taken to measure the accuracy of the surrogates. It was determined that the average error at 
these twenty test points was less than 0.03% for all of the geometry/flow type/loudness quantity combinations 
indicating the selection of a second-order PCE was sufficient. Note that these test points are different from 
those points used to train the surrogate models and are distributed evenly throughout the design space with 
a Latin Hypercube sample structure. 

With the surrogate models constructed and validated, the uncertainty can be propagated through the 
surrogates using the second-order probability analysis outlined in section IV. This was done using the 
sampling approach, resulting in the family of CDFs. For the Euler as-built case, the boundaries of the 
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P-boxes are shown in Figure 11(a) and Figure 11(b) for PLdB and CSEL, respectively. 




Figure 11: SEEB-ALR Euler as-Built PLdB and CSEL Probability Box Boundaries 

A significant amount of information can be disseminated from P-boxes, including confidence intervals. 
For the case of mixed uncertainty, one approach to obtaining the 95% confidence interval, for example, is to 
take the upper 97.5% probability level and the lower 2.5% probability level as the interval. A summary of 
the 95% confidence intervals is shown in Table 5 for all of the models, subject to the uncertainty identified 
in this study. 


Table 5: SEEB-ALR 95% Confidence Intervals 


Configuration 

PLdB 

CSEL 

Euler as-Built 
Euler as-Designed 
Turbulent as-Built 
Turbulent as-Designed 

[89.12 , 91.63] 
[88.06 , 90.49] 
[89.44 , 91.95] 
[88.98 , 91.61] 

[94.64 , 96.05] 
[94.32 , 95.80] 
[94.78 , 96.22] 
[94.75 , 96.20] 


From these results, it can been seen that there is a difference between the ideal as-designed geometry 
and the actual as-built model. The imperfections in the surface cause an increase in the PLdB level. In 
the case of CSEL, the same trend is not as severe, but still present. Additionally, it can be disseminated 
that the effect of an inviscid versus the fully turbulent analysis is small and is less than that of the effect of 
the surface imperfections in the as-built geometry. For preliminary design and analysis, this indicates the 
use of the low fidelity solution may be acceptable when considering the added computational cost of a fully 
turbulent solution, for this configuration. 

3. Certification Prediction 

The final step in the analysis is to predict whether or not the design, under uncertainty, could potentially pass 
a certification based on the design quantity of interest. Currently, the actual certification value for low-boom 
configurations is unknown. However, the methodology outlined in section V can be easily implemented for 
certification prediction. For example, if the certification requirement was a PLdB of less than 100, the Euler 
as-built design would have a CR of 4.27 for a 95% confidence analysis. This may indicate that the margin 
is sufficiently larger than the uncertainty in the design. 
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4- Sensitivity Analysis 

A sensitivity analysis of the uncertainty parameters was performed to highlight which of the parameters 
contribute most significantly to the total uncertainty. The contribution of the top uncertain parameters to 
both PLdB and CSEL are given in Tables 6 and 7, respectively. For both metrics, the reflection factor appears 
to be the most dominate of the parameters considered. This is especially true for CSEL as the reflection factor 
contributes about 85 to 90% to the total output variance. For PLdB, a significant amount of uncertainty also 
arises due to the uncertainty in the atmospheric humidity profile. Loubeau and Coulouvrat 41 have shown 
that variation in the atmospheric humidity can significantly effect the sonic-boom rise time, which may effect 
the signature loudness. 

Table 6: SEEB-ALR Top Uncertain Parameter Contribution to PLdB Total Uncertainty 


Uncertain Parameter 

Euler 

as-Built 

Euler 

as-Designed 

Turbulent 

as-Built 

Turbulent 

as-Designed 

Angle of Attack 

4.7% 

9.6% 

2.4% 

6.7% 

Initial Step Size 

1.6% 

1.1% 

1.7% 

1.8% 

Reflection Factor 

46.4% 

44.8% 

45.9% 

44.2% 

Humidity Profile 

38.3% 

35.7% 

41.6% 

36.1% 

Ground Elevation 

7.9% 

7.7% 

6.8% 

9.7% 

All Others 

<1% 

<1% 

<1% 

<1% 


Table 7: SEEB-ALR Top Uncertain Parameter Contribution to CSEL Total Uncertainty 


Uncertain Parameter 

Euler 

as-Built 

Euler 

as-Designed 

Turbulent 

as-Built 

Turbulent 

as-Designed 

Angle of Attack 

3.6% 

6.2% 

4.5% 

4.6% 

Reflection Factor 

88.2% 

84.1% 

86.5% 

86.0% 

Temperature Profile 

2.2% 

2.4% 

2.4% 

2.4% 

Humidity Profile 

1.7% 

1.5% 

1.7% 

1.7% 

Ground Elevation 

4.1% 

5.5% 

4.6% 

5.2% 

All Others 

<1% 

<1% 

<1% 

<1% 


C. NASA 69° Delta Wing 

1. Comparison with Experiment 

For the NASA 69° Delta Wing, near-field signatures were taken at 24.8 inches from the body to coincide 
with experimental data. Comparison with the experiment is shown in Figure 12(a) for both inviscid and 
fully turbulent flow assumptions. A residual scale plot of the signatures is shown in Figure 12(b). The CFD 
signatures are in fairly good agreement with the experimental results as the signatures lie primarily within 
the bounds of the uncertainty in the experimental results. There is some rounding of the experimental results 
similar to the results of the SEEB-ALR model. This is due to model vibration and the instrumentation used 
during the wind tunnel testing. 3 ' However, given these small and expected differences, the baseline numerical 
solution is considered to be accurate and validated for the purposes of this study. 

2. Uncertainty Quantification 

As above, surrogate models are constructed for the design quantities of interest, PLdB and CSEL. The 
number of uncertain parameters is the same as with the SEEB-ALR and, therefore, the deterministic model 
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Delta Wing h=24.8 in, phi=0 Delta Wing h=24.8 in, phi=0 




(a) Signatures (b) Residual Scale 

Figure 12: 69° Delta Wing Near-Field CFD Comparison with Experimental Results 


is evaluated 182 times to construct a second order PCE. The dispersion of the deterministic samples for 
the Euler case are shown in Figure 13(a). Figure 13(b) shows the same dispersion of signals, but shifted to 
reference location for better visualization of changes in signature amplitude. The resulting ground signature 
dispersion, for the Euler case, is shown in Figure 14(a) and shifted to a reference location in Figure 14(b). 
Similar dispersions of the near-field and ground level signatures can be generated for the fully turbulent case. 
While not shown here, this dispersions are similar to those shown above for the Euler case. 




(a) Actual (b) Shifted 

Figure 13: 69° Delta Wing Euler Near-Field Pressure Signature Dispersion 

As with the SEEB-ALR, the accuracy of the surrogate model needs to be verified. It was determined 
that the average error at the 20 test points was less than 0.4% for all of the flow type/loudness quantity 
combinations indicating the selection of a second-order PCE was sufficient. With the surrogate models 
constructed and validated, the uncertainty can be propagated through the surrogates using the second-order 
probability analysis outlined in section IV. This was done using the sampling approach, resulting in the 
family of CDFs. For the Euler case, the boundaries of the probability boxes are shown in Figure 15(a) and 
Figure 15(b) for PLdB and CSEL, respectively. 

A summary of the 95% confidence intervals for both Euler and Turbulent cases is shown in Table 8. From 
these results, it can be seen that the effect of an inviscid versus the fully turbulent analysis is slightly more 
significant compared to the SEEB-ALR model for both loudness measures. However, there is still less than 
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(a) Actual (b) Shifted 

Figure 14: 69° Delta Wing Euler Ground Signature Dispersion 




Figure 15: 69° Delta Wing Euler PLdB and CSEL Probability Box Boundaries 


a one PLdB and CSEL difference between the two levels of model fidelity. This may indicate that the use of 
a Euler analysis may be sufficient given the reduce computational cost over the fully turbulent analysis, for 
this model. 


Table 8: 69° Delta Wing 95% Confidence Intervals 


Configuration 

PLdB 

CSEL 

Euler 

[93.16 . 

, 95.58] 

[97.18 , 

, 98.46] 

Turbulent 

[94.03 . 

, 96.35] 

[97.63 . 

, 98.85] 


3. Sensitivity Analysis 

The contribution of the top uncertain parameters to both PLdB and CSEL are given in Tables 9 and 10, 
respectively. The results of this sensitivity analysis are similar to those observed for the SEEB-ALR. The 
reflection factor contributes even more to the total output uncertainty for both loudness metrics, and the 
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contribution of the atmospheric humidity variation contributes significantly to PLdB. 


Table 9: 69° Delta Wing Top Uncertain Pa- Table 10: 69° Delta Wing Top Uncertain Pa- 
rameter Contribution to PLdB Total Uncer- rameter Contribution to CSEL Total Uncer- 
tainty tainty 


Uncertain Parameter 

Euler 

Turbulent 

Uncertain Parameter 

Euler 

Turbulent 

Initial Step Size 

1.4% 

1.0% 

Reflection Factor 

93.1% 

94.4% 

Reflection Factor 

50.9% 

52.0% 

Temperature Profile 

2.1% 

2.5% 

Temperature Profile 

1.3% 

1.8% 

Humidity Profile 

1.1% 

1.5% 

Humidity Profile 

37.1% 

38.0% 

Ground Elevation 

1.9% 

1.4% 

Ground Elevation 

7.9% 

6.3% 

All Others 

<1% 

<1% 

All Others 

<1% 

<1% 





D. LM 1021-01 Low-Boom Configuration 

1. Comparison with Experiment 

For the LM 1021-01, near-field signatures were taken at 20.7 inches from the body to coincide with experi- 
mental data. Comparison with the experiment is shown in Figure 16(a) for both inviscid and fully turbulent 
flow assumptions. A residual scale plot of the signatures is shown in Figure 16(b). The fully turbulent 
solution agrees well with the experimental results, with the exception of the rounding of the peaks in the 
experimental results, similar to the delta wing and SEEB-ALR models. The inviscid solution, however, 
does not agree as well. Aftosmis et al. 18 identify the source of this discrepancy, which stems from a shock 
originating at the front of the under wing nacelle. In the viscous case, this shock is smeared by the boundary 
layer between the wing and nacelle, but propagates in the inviscid simulation. A more complete discussion 
is included in the reference. An important note should be made regarding the length of the LM 1021-01 
signature. The computational domain is slightly truncated as the signature does not fully recover to zero. 
As stated above, the ends of the signatures are forced to zero to prevent any numerical issues in propagating 
the signature to the ground level. This artificial forcing of the signature, however, is not expected to effect 
the loudness results as this is not in a strong shock or expansion region. 


LM1 021 h=20.7 in, phi=0 LM1 021 h=20.7 in, phi=0 




(a) Signatures (b) Residual Scale 

Figure 16: LM 1021-01 Near-Field CFD Comparison with Experimental Results 
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2. Uncertainty Quantification 


The surrogate models for CSEL and PLdB are constructed in the same fashion as for the other cases. The 
dispersion of the deterministic samples for the Euler case are shown in Figure 17(a). Figure 17(b) shows the 
same dispersion of signals, but shifted to reference location for better visualization of changes in signature 
amplitude. For the Euler case, the resulting ground signature dispersion is shown in Figure 18(a) and shifted 
to a reference location in Figure 18(b). The ground signatures were then analyzed to produce the desired 
loudness quantities of interest. 




X (ft) X (ft) 

(a) Actual (b) Shifted 

Figure 17: LM 1021-01 Euler Near-Field Pressure Signature Dispersion 




(a) Actual (b) Shifted 

Figure 18: LM 1021-01 Euler Ground Signature Dispersion 

Similar dispersions of the near-field and ground level signatures can be generated for the fully turbulent 
case. Because of the effect of different flow physics modeling, the signatures for the turbulent case are 
slightly different, both in the near-field, as well as at the ground level. The near-field signatures are shown 
in Figure 19(a) and shifted to a reference location in Figure 19(b). The ground level signatures are shown 
in Figure 20(a) and shifted to a reference location in Figure 20(b). 

The surrogates are validated using 20 test points distributed throughout the design space, similar to 
the previous two models. The average error at these 20 test points was less than 0.2% for all of the flow 
type/loudness quantity combinations indicating the selection of a second-order PCE was sufficient. The 
uncertainty is propagated through the surrogates and a family of CDFs were produced. For the Euler case, 
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(a) Actual (b) Shifted 

Figure 19: LM 1021-01 Turbulent Near-Field Pressure Signature Dispersion 




(a) Actual (b) Shifted 

Figure 20: LM 1021-01 Turbulent Ground Signature Dispersion 


the boundaries of the probability boxes are shown in Figure 21(a) and Figure 21(b) for PLdB and CSEL, 
respectively. 

A summary of the 95% confidence intervals for both Euler and Turbulent cases is shown in Table 11. 
From these results, it can be seen that the effect of an inviscid versus the fully turbulent analysis is more 
significant than for the SEEB-ALR and the Delta wing models for both loudness measures. This is due 
to the reduced order modeling fidelity of the Euler analysis. While the loudness values are lower for the 
inviscid case, which is desirable, the accuracy of the Euler model has to be in question due to this significant 
difference. Along with the discrepancy in the near-field signature, this indicates an inviscid assumption is not 
appropriate for this configuration under the current flow conditions. Aftosmis et al. 18 notes the wind tunnel 
experiments were conducted at a relatively low Reynolds number (which was matched for the simulations). 
At a higher Reynolds number, an Euler solution may be sufficient, but this must first be validated for the 
new freestream conditions. 

3. Sensitivity Analysis 

The contribution of the top uncertain parameters to both PLdB and CSEL are given in Tables 12 and 
13, respectively. The results of this sensitivity analysis is different than that observed for the SEEB-ALR 
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Figure 21: LM 1021-01 Euler PLdB Probability Box Boundaries 


Table 11: LM 1021-01 95% Confidence Intervals 


Configuration 

PLdB 

CSEL 

Euler 

[87.76 , 90.60] 

[94.43 , 96.85] 

Turbulent 

[90.17 , 93.79] 

[96.06 , 98.76] 


and Delta Wing models as the reflection factor is no long the top parameter. While the reflection is still 
significant, angle of attack now dominates the output uncertainty for both loudness metrics. Because of the 
specific design of the LM 1021-01, changes in angle attack may effect the use of specific design features, 
especially those related to shock cancellation off the body. 


Table 12: LM 1021-01 Top Uncertain Param- Table 13: LM 1021-01 Top Uncertain Param- 
eter Contribution to PLdB Total Uncertainty eter Contribution to CSEL Total Uncertainty 


Uncertain Parameter 

Euler 

Turbulent 

Uncertain Parameter 

Euler 

Turbulent 

Mach Number 

1.4% 

0.2% 

Angle of Attack 

57.2% 

63.9% 

Angle of Attack 

39.0% 

55.1% 

Reflection Factor 

38.2% 

32.0% 

Reflection Factor 

33.8% 

21.9% 

Temperature Profile 

1.1% 

1.3% 

Temperature Profile 

1.6% 

0.7% 

Ground Elevation 

2.4% 

2.9% 

Humidity Profile 

22.7% 

17.9% 

All Others 

<1% 

<1% 

Ground Elevation 

1.6% 

4.3% 




All Others 

<1% 

<1% 





VIII. Conclusions 

The objective of this work was to develop and demonstrate a non-intrusive, computationally efficient 
approach to uncertainty quantification and certification prediction for low-boom, supersonic aircraft config- 
urations. Key components of the developed framework included the use of a surrogate based uncertainty 
quantification approach using non-intrusive polynomial chaos, the propagation of mixed aleatory and epis- 
temic uncertainties, and a measure of the margin and uncertainty between the design, under uncertainty, and 
a certification value used for certification prediction. The use of a surrogate based, non-intrusive approach 
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was implemented for computationally efficiency and ease of implementation when investigating the large- 
scale, high-fidelity computational fluid dynamics models analyzed in this study. The non-intrusive nature 
of the approach presented here is advantageous because of its broad applicability to any type of complex, 
multiphysics simulation. 

This process was successfully demonstrated on three low-boom configurations: the Lockheed Martin 
SEEB-ALR body of revolution, the NASA 69° Delta Wing, and the Lockheed Martin 1021-01 configuration. 
Sources of uncertainty considered in the high-fidelity, multiphysics models included variability in freestream 
conditions, atmospheric profiles, and other modeling parameters used in the propagation of the sonic boom 
signatures from the near-field to the ground level. Sensitivity analyses were also performed to identify 
important uncertain parameters. Additionally, the use of an inviscid versus a fully turbulent flow assumption 
was investigated for each configuration. 

The uncertain inputs and subsequent results given in this study were only the finalized results. There were 
multiple intermediate steps before finalizing uncertain inputs and even the numerical models. One of the key 
advantages of the process and methodologies outlined in this study, is its applicability in efficiently identifying 
significant sources of uncertainty and their effect on output uncertainty. Input parameters that contribute 
significantly to the uncertainty may be subject of refinement, model improvement, or may help identify 
incorrect models/parameters. For example, in performing this study there was a significant intermediate 
change in the numerical approached used to calculate the loudness values from the ground level signature. 
Initial results indicated that the sBOOM sampling rate input parameter contributed as much as 80% to the 
total output uncertainty. More importantly, 95% confidence interval widths for PLdB and CSEL were as 
much as an order of magnitude greater than the results reported above. This led to further investigation, 
which resulted in two changes to the analysis. First, the dependence of the loudness metrics on the number 
of sBOOM sample points was better quantified and slight refinement was made to the epistemic uncertainty 
range. Second, the information provided in these intermediate results helped identify an issue within the 
numerical routine used to integrate ground level signature to obtain the loudness metrics. Correcting this 
issue led to the results detailed throughout this study. 
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